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(54) Network service server load balancing 



(57) For use with a system in which data is trans- 
ferred from a first control unit (5-1 ) to a plurality of sec- 
ond control units (1-1, 1-2, .... 1-m) over a network (2), 
a network load balancing device (3) comprises: a con- 
version information storage unit (4-a) which stores con- 
version information representing a correspondence re- 
lationship of the first control unit (5-1 ) to one of the sec- 
ond control units (1-1, 1-2, 1-m); a distributive relay 
unit (4) which transfers data sent by the first control unit 
(5-1) to one or more of the second control units (1-1, 



1-2, 1-m) on the basis of the correspondence rela- 
tionship stored in the conversion information storage 
unit (4-a); a distribution ratio adjusting unit (7) which ad- 
justs the correspondence relationship in the conversion 
information stored in the conversion information storage 
unit (4-a); and a load measuring unit (6) which measures 
the load on each of the second control units (1-1, 1-2, 
1 -m) and notifies the distribution ratio adjusting unit (7) 
of the measurements to allow the unit to adjust the cor- 
respondence relationship in the conversion information 
stored in the conversion informatbn storage unit (4-a). 
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Description 

[0001] The present invention relates to the balancing 
of the service load on a network service server. 
[0002] More specifically, the present invention relates s 
to a network service server load balancing device, a net- 
work service server load balancing method, and a com- 
puter-readable storage medium recorded with a net- 
work service server load balancing program, which al- 
low the network service load to be distributed dynami- io 
cally and efficiently to each of multiple network service 
servers according to its status. 

[0003] In recent years, as network services have in- 
creased with the rapid spread of Internet/Intranet, the 
demand has increased for improving the performance is 
of and permitting more efficient utilization of the client 
server system and increasing the stability of services of 
servers. In particular, there is a demand for an environ- 
ment which permits centralized access to the WWW 
(World Wide Web) server to be circumvented and fail- 20 
ures to be hidden. For this reason, a system has been 
provided which uses two or more servers (or nodes) to 
perform one service (for example, ftp (file transfer pro- 
tocol), HTTP (Hyper Text Transfer Protocol), telnet, or 
the like). 25 
[0004] In order to implement services with stability, it 
is required to distribute services to each server suitably. 
On the other hand, the network services have become 
increasingly diversified, complicated, and advanced, 
and the frequency at which changes are made to the 30 
configuration of a group of servers and the service dis- 
tribution method has increased. Also, the demand has 
increased for circumventing stops of some services due 
to some servers going down unexpectedly. 
[0005] Existing techniques of distributing services to 35 
multiple servers include: 

(a) Round-robin DNS (Domain Name Server): in the 
DNS service, an entry table is set up in which mul- 
tiple server IP (Internet Protocol) addresses are *o 
mapped to one domain name. When an inquiry is 
made by a client about a server IP address, servers 
are allocated to the client on a round robin basis 
according the entry table and the IP addresses of 

the allocated servers are presented to the client, 45 
thereby distributing services to multiple servers. 

(b) Load distribution hardware: a hardware devic e 
i s_ placed between a server group and a network to 
relay communications between clients and server s. 
Load measu ring communications are made be- so 
tween the hardware device and each server and 
packets to oe relayed are monitored to measure the 

STOr^eTB T ^thereTpv detecting the load condition. 
o rfeacn server and distributing services to the serv - ~ss 

s accordingly^ "~ ~ — — ' 
r (c) Agent: a functbn, called Agent, which is reside 
on each server in a server group measures the load 



on its CPU (Central Processing Unit) and its disk 
utilization to see its load condition. The load distri- 
bution system is notified of the load condition of 
each server and distributes services to the servers 
accordingly. 

[0006] However the conventional techniques have 
the following problems: 

[0007] Round robin DNS: in this system, services are 
distributed to servers only equally or at simple rates and 
each server has thus to perform services allocated to 
itself irrespective of its capabilities and dynamic load 
conditions. This produces a difference in load condition 
between each server resulting in reduced efficiency of 
the whole system. Further, in the event that a server has 
gone down and the configuration of the server group has 
to be modified accordingly, it is required to make man- 
ually such a change to the server group configuration 
as deletes a server that went down from the entry table. 
This change is made each time a server goes down. It 
is therefore difficult to cope with such a situation imme- 
diately. As a result, the whole system will have to be 
stopped temporarily. 

[0008] Load distribution hardware: this system uses 
hardware and hence its introduction costs much. The 
employment of this system is limited because the hard- 
ware will not be incorporated into each server. In addi- 
tion, since communications for load measurement are 
always needed between each server, extra load, which 
is different from original communications, is imposed on 
each server, which further increases traffic and may 
cause servers to go down. Furthermore, since the load 
is measured on a packet-by-packet basis, the servers 
may be switched even in mid-service and consequently 
errors may occur. 

[0009] Agent: since the agent function resides on 
each server, the server has to be modified at the time 
the function is installed and moreover the agent must 
conform to the server's operating system (OS). In addi- 
tion, the load measurement is always made for each 
server, resulting in a further increase in the load on the 
server. Furthermore, as is the case with the load distri- 
bution system, since the load is measured on a packet- 
by-packet basis, the servers may be switched even in 
mid-service and consequently errors may occur. 
[0010] Embodiments of the present invention to pro- 
vide a network service server load balancing device, a 
network service server load balancing method, and a 
computer-readable recording medium recorded with a 
network service server load balancing program, which 
permit changes in the server group configuration and 
server statuses to be accommodated instantly, permit 
the changes to be reflected on a client instantly, require 
no modification of the servers at installation time, and 
permit various forms of employment to be accommodat- 
ed flexibly. 

[0011] A network service server load measurement 
device of the present invention comprises a conversion 
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information storage unit, a distributive relay unit, a dis- 
tribution ratio adjusting unit, a load measuring unit, a first 
control unit, second control units, a third control unit, a 
storage unit, and an adjustment unit. 
[001 2] According to a first aspect of the present inven- 
tion, the conversion information storage units stores 
conversion information representing a correspondence 
relationship of the first control unit to the second control 
units. The distributive relay unit transfers data sent by 
the first control unit to one or more of the second control 
units on the basis of the conversion information stored 
in the conversion information storage unit. The distribu- 
tion ratio adjusting unit adjusts the correspondence re- 
lationship in the conversion information stored in the 
conversion information storage unit. The load measur- 
ing unit measures loads on the second control units and 
notifying the distribution ratio adjusting unit of load con- 
ditions of the second control units to thereby allow the 
distribution ratio adjusting unit to adjust the correspond- 
ence relationship in the conversion information stored 
in the conversion information storage unit. 
[0013] According to a second aspect of the present 
invention, the first control unit sends data. The second 
control units receive the data. The third control unit 
transfers the data sent by the first control unit to the sec- 
ond control units. The conversion information storage 
unit stores conversion information which represents a 
correspondence relationship of the first control unit to 
the second control units. The distributive relay unit 
transfers the data sent by the first control unit to one or 
more of the second control units according to the corre- 
spondence relationship in the conversion information 
stored in the conversion information storage unit. The 
distribution ratio adjusting unit adjusts the correspond- 
ence relationship in the conversion information stored 
in the conversion information storage unit. The load 
measuring unit measures the load on each of the sec- 
ond control units and notifying the distribution ratio ad- 
justing unit of load conditions of the second control units 
in order to allow the distribution ratio adjusting unit to 
adjust the correspondence relationship in the conver- 
sion information stored in the conversion information 
storage unit. 

[0014] According to a third aspect of the present in- 
vention, the conversion information storage unit stores 
conversion information which represents a correspond- 
ence relationship of the first control unit to the second 
control units. The distributive relay unit transfers the da- 
ta sent by the first control unit to one or more of the sec- 
ond control units according to the correspondence rela- 
tionship in the conversion information stored in the con- 
version information storage unit. The distribution ratio 
adjusting unit adjusts the correspondence relationship 
in the conversion information stored in the conversion 
information storage unit. The load measuring unit meas- 
ures the load on each of the second control units and 
notifying the distribution ratio adjusting unit of load con- 
ditions of the second control units in order to allow the 



distribution ratio adjusting unit to adjust the correspond- 
ence relationship in the conversion information stored 
in the conversion information storage unit. 
[0015] According to a fourth aspect of the present in- 
5 vention, the conversion information storage unit stores 
conversion information which represents a correspond- 
ence relationship of the first control unit to the second 
control units. The distributive relay unit transfers the da- 
ta sent by the first control unit to one or more of the sec- 
10 ond control units according to the correspondence rela- 
tionship in the conversion information stored in the con- 
version information storage unit. 
[001 6] According to a fifth aspect of the present inven- 
tion, the storage unit stores relevant data so that it will 
15 be processed in a corresponding server The balancing 
unit decides whether each server is over loaded and, if 
it is, balances the load on that server so that it will be 
distributed to the other servers. 
[0017] The present invention will be more apparent 
from the following detailed description, when taken in 
conjunction with the accompanying drawings, in which: 

FIG. 1 is a schematic representation of the present 
invention; 

FIG. 2 is a block diagram of a system configuration 
of the present invention; 

FIG. 3A shows an example of hash function-based 
distribution of services to servers; 
FIG. 3B shows another example of hash function- 
based distribution ol services to servers; 
FIG. 4 shows an arrangement of the conversion ta- 
ble shown in FIG. 2; 

FIG. 5 is a flowchart illustrating to which server a 
packet is transferred; 

FIG. 6 shows an example of a pattern hash table; 
FIG. 7 is a diagram for use in explanation of the use 
of the pattern hash table having a VCP list; 
FIG. 8 is a diagram for use in explanation of the use 
of the pattern hash table having a VCP list; 
FIG. g is a diagram for use in explanation of the use 
of the pattern hash table having a VCP list; 
FIG. 10 is a diagram for use in explanation of the 
use of the pattern hash table having a VCP list; 
FIG. 11 is a flowchart for load measurement; 
FIGs. 1 2 and 1 3 are a flowchart for load balancing; 
FIG. 14 is a flowchart for judgment of over load; 
FIG. 1 5 A is a diagram for use in explanation of over- 
load judgment based on the average connection 
time; 

FIG. 1 5B isa diagram for use in explanation of over- 
load judgment based on the number of connections; 
FIG. 16 is a diagram for use in explanation of a var- 
iation in the number of slots; 
FIG . 1 7 is a flowchart for determination of upper and 
lower limit values for the number of slots; 
FIG. 18 is a flowchart for judgment of "server stop" 
due to over load; 

FIG. 19 is a diagram for use in explanation of vari- 
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ations in the number of slots; 
FIG. 20 is a flowchart for judgment of over load; 
FIG. 21 is a flowchart for judgment of over load; 
FIG. 22 is a block diagram for load measurement 
using a dummy client; 

FIG. 23 is a flowchart for judgment of over load; 
FIG. 24 is a flowchart for judgment of over load; 
FIG. 25 is a flowchart lor determining servers whose 
measured values differ from the average measured 
value by an arbitrary value; 
FIG. 26 is a diagram for use in explanation of vari- 
ations in the number of slots; 
FIG. 27 is a flowchart for allocating the slots to each 
server according to the conversion table; 
FIG. 28 is a block diagram of the failure monitor of 
FIG. 2; 

FIG. 29 A shows another system configuration; 
FIG. 29B shows still another system configuration; 
FIG. 29C shows a further system configuration; and 
FIG. 30 shows an implementation of the present in- 
vention using a recording medium. 

[0018] FIG. 1 is a conceptual diagram of the present 
invention. 

[001 9] The inventive device has a group address, vip, 
that represents multiple servers, and a client sends 
packets to vip. Packet transfer is made by changing the 
destination of the packets from the vip to the server's 
address. The main operation of packet transfer includes 
receiving a packet, masking its header to obtain a pat- 
tern (mask(head) -» pat), converting the pattern, pat, in- 
to an index in a conversion table using a hash function 
(hash(pat) ix), changing the destination of the packet 
from vip to ip where ip is a destination address corre- 
sponding to the address of a server that a slot, ix, in the 
conversion table has, and sending the packet to that 
server. For load measurement, the connection time is 
measured through processing of sending packets to the 
server. A total quantity of connection time is computed 
from the start of the measurement (the previous meas- 
ured values are reset), and the number of connections 
is incremented. 

[0020] The network service server load balancing de- 
vice of the present invention distributes packets from a 
client to servers in accordance with the contents of the 
conversion table. The conversion table is updated so 
that the distribution ratio of packets to each server will 
be converged dynamically to the optimum one accord- 
ing to the load and failure conditions of the respective 
servers. 

[0021] FIG. 2 is a block diagram of a system configu- 
ration of the present invention. 
[0022] A first embodiment of the present invention is 
directed to a network load balancing device which, for 
use with a system in which data is transferred from a 
first control unit to a plurality of second control units over 
a network, comprises a conversion information storage 
unit which stores conversion information that represents 



a correspondence relationship of the first control unit to 
the second control units, a distributive relay unit which 
transfers data from the first control unit to one or more 
of the second control units on the basis of the corre- 

5 spondence relationship stored in the conversion infor- 
mation storage unit, a distribution ratio adjusting unit 
which adjusts the correspondence relationship informa- 
tion stored in the conversion information storage unit, 
and a load measurement unit which measures the load 

10 conditions of the respective second control units and no- 
tifies the distribution ratio adjusting unit of the measured 
load conditions to allow the balancing unit to balance 
the correspondence relationship of the first control unit 
to the second control units. 

is [0023] In FIG. 2, reference numerals 1-1, 1-2, .., 1-m 
denote client computers available to users, which re- 
ceive services from server computers 5-1, 5-2,..., 5-n 
over a network 2 and a network service server load bal- 
ancing device 3. The network 2 is the Internet, an intran- 

20 et, a local area network (LAN), or a wide area network 
(WAN). Reference numeral 11 denotes a network which 
is made up by a group of the server computers 5-1 , 
5-2... 5-n. Reference numeral 3 denotes a network 
service server load balancing device which is composed 

25 of a distributive relay unit 4, a conversion table 4-a, a 
load measurement unit 6, a distribution ratio adjusting 
unit 7, a failure monitor unit 8, a control unit 9, and a 
status management unit 10. The balancing unit 3 has 
virtual addresses vip_1 , vip_2, . . . , vip_1 and, when a cli- 

30 ent makes access to the address vip_i (1 < i < 1 ), allows 
the servers 5-1 , 5-2,..., 5-n to look like a single computer 
having the address vipj. The distributive relay unit 4 
relays communications between the clients 1-1, 1-2,..., 
1-m and the servers 5-1, 5-2 ... 5-n. More specifically, 

35 the relay unit receives a packet having the destination 
address vip i from one of the clients, masks its header 
(containing the IP address of the client itself, a port 
number representing the type of a service such as ftp, 
HTTP, telnet, or the like, the IP address of the destina- 

40 tion server, etc.) to obtain a pattern, identifies structured 
data having the same pattern in the conversion table 4-a 
for vip_i by means of pattern matching, converts the pat- 
tern to an index to identify a slot, determines a server to 
which the packet is to be directed from server informa- 

45 tion that slot holds, and sends the packet to the server 
thus determined. The details of the conversion table 4-a 
will be described later. The load measurement unit 6 
measures the load on each of the servers. More specif- 
ically, the load measuring unit monitors packets the dis- 

so tributive relay unit relays, measures the load conditions, 
such as the average connection time and the number of 
connections per unit time, for each of the servers, and 
presents the measurements to the distribution ratio ad- 
justing unit 7 at regular intervals. The balancing unit de- 

ss tects the stop of a server depending on whether its 
measurement indicates an abnormal value and detects 
over loaded servers using a reference value. In re- 
sponse to stopped-server notification from the failure 
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monitor unit 8 or an instruction from the control unit 9, 
the balancing unit recomputes the service distribution 
ratio when over-doped servers or stopped servers are 
detected or as instructed by the user and updates the 
conversion table 4-a accordingly. The failure monitor 
unit 8 performs a polling command, such as ping, on 
each server at regular intervals and considers servers 
that caused a certain number of time-outs to be in the 
stopped state. This information is presented to the dis- 
tribution ratio adjusting unit 7. The control unit 9 controls 
the network service server load balancing unit 3 in its 
entirety as instructed by the user. More specifically, the 
control unit starts or stops the load measurement unit 6, 
the distribution ratio adjusting unit 7, and the failure 
monitor unit 8 and retains and restores the information 
set in the conversion table 4-a. The status management 
unit 10 retains information set in the conversion table 
4-a and the network service server balancing unit 3 in a 
file and restores it. More specifically, the status manage- 
ment unit retains information set in the conversion table 
4-a and the network service server load balancing unit 
3 in a file when the distribution ratio adjusting unit is 
stopped and additions are made to servers and services 
or when a command is given by the user. When the dis- 
tribution ratio adjusting unit is started and the servers or 
services are restored or when a command is given by 
the user, the status management unit restores the infor- 
mation, provided that the server group configuration is 
the same as before the distribution ratio adjusting unit 
is stopped or before the servers or services are stopped. 
[0024] Since the load conditions of the servers that 
make up a network are measured and packets from cli- 
ents are sent to the servers according to the load con- 
ditions of each client, well-balanced, efficient services 
can be provided. Moreover, since it is known which serv- 
er a sequence of packets is being sent to, no server 
switching occurs in the middle of a sequence of servic- 
es. Furthermore, since there is no need of flowing data 
dedicated to load balancing on communication paths, 
the paths can be employed for services only. 
[0025] How to distribute packets sent by the clients to 
the servers in the above arrangement and how to con- 
verge the distribution ratio of the packets to the servers 
dynamically to the optimum state will be described sep- 
arately. 

[0026] A second embodiment of the present invention 
is directed to a network load balancing device charac- 
terized in that the load measuring unit measures the 
load on the basis of the average time of connecting data 
to each second control unit and the number of connec- 
tions of data to the second control unit per unit time. 
[0027] First, how to distribute packets sent by clients 
to the servers will be described. 
[0028] As shown in FIGs. 3A and 3B, if a hash function 
can distribute indexes uniformly, then the service distri- 
bution ratio to the servers becomes equal to the ratio of 
the numbers of slots assigned to the servers in the con- 
version table. 



[0029] As an example, if, as shown in FIG. 3A, the 
conversion table has 256 slots (numbered 0 to 255), and 
server 1 and server 2 are assigned 64 slots and 192 
slots, respectively (consecutive slots need not neces- 
s sarily be assigned to each server), then services will be 
distributed to server 1 and server 2 at a ratio of 1 : 3. 
Thus, the service distribution ratio can be changed by 
changing the number of slots assigned to each server. 
If, as shown in FIG. 3B, 64 slots of 192 slots assigned 
to server 2 are assigned to server 1 , then the ratio of the 
number of slots assigned to server 1 to that assigned to 
server 2 will become one to one (1:1) and the service 
distribution ratio will also become one to one (1:1). 
[0030] FIG. 4 shows an arrangement of the conver- 
sion table. 

[0031] The conversion table 4-a is one referred to as 
a pattern hash table 21 that is structured data composed 
of one or more slots each accessible by an index ob- 
tained by applying a hash function to a pattern obtained 
by masking a packet header. Data additions or deletions 
can be made to or from the conversion table at the time 
of system operation. Each slot contains information 
which can identify one of the servers 5-1 . 5-2,..., 5-n and 
a VCP (Virtual Circuit Preservation) list 22 which holds 
client-server information in a service being relayed 
through that slot. Since the service distribution ratio can 
be controlled by the ratio of numbers of slots assigned, 
it follows that the more slots a server is assigned, the 
more services it will be assigned. 
[0032] FIG. 5 is a flowchart illustrating which server a 
packet is transferred to. 

[0033] First, in step S301 , the network service server 
load balancing unit 3 receives a packet sent from a cli- 
ent. In step S302, the distributive relay unit 4 obtains a 
pattern of the header of the received packet. The pattern 
of the header is a set of bit streams obtained by masking 
the packet header and takes the form of, for example, 
(cip_1, cpo_1, 23). Here, cip_1 is the client IP address, 
cpo_1 is the client port number, and 23 is the service 
port number. In step S303, the distributive relay unit 4 
searches through the pattern hash table in the conver- 
sion table 4-a for a pattern that matches the pattern 
(cip_1 , cpo_1 , 23). For example, such a pattern (*, *, 23) 
as shown in FIG. 6 is obtained. This pattern matches a 
pattern of an arbitrary IP address, an arbitrary port 
number, and the service port number 23. Next, in step 
S304, a hash function is applied to the pattern obtained 
in step S302. In step S305, the solution is obtained. For 
the hash function, the remainder of the division by 8 is 
set as the solution. It is assumed here that the solution 
[3] was obtained. This means that slot 3 was selected 
in FIG. 6. In slot 3, server 5-1 is set as server informa- 
tion. 

[0034] Next, in step S306, a decision is made as to 
whether the received packet is the first one of a se- 
quence of packets. A sequence ol packets is packets in 
which a client having the same client IP address cip_1 
requests the same client port cpo_1 a for a service of the 
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same service port number 23. Whether the received 
packet is the first one of a sequence of packets depends 
on whether the VCP list of slot 3 contains VCP (cip_1 , 
cpo_1 , 5-1 ). If that packet is the first one, the VCP list 
22 does not contain VCP (cip_1, cpo_1, 5-1). In step 
S307, therefore, VCP (cip_J, cpo_1, 5-1) is created in 
the VCP list 22. In step S308, the created packet is sent 
to the server 5-1 . If, in step S306, the VCP list contains 
VCP (cip_1, cpo_1, 5-1), that is, if packets in which a 
client having the same client IP address cip_1 requests 
the same client port cpo_1 for a service of the same 
service port number 23 are being transferred, that pack- 
et is transferred to the server 5-1 identified by VCP 
(cip_1 , cpo_1 , 5-1 ) in the VCP list 22. 
[0035] Next, in step S310, a decision is made as to 
whether this packet is the last one. If it is, VCP (cip_1 , 
cpo_1 , 5-1 ) is deleted from the VCP list 22. 
[0036] Reference will be made to FIGs. 5 through 10 
to describe how four packets will be transferred to a 
server in a state where no packet is received. To simplify 
the description, it is assumed that these packets con- 
tains the same client IP address cip_1, the same client 
port cpo_1 , and the same service port number 23. Steps 
S301 to S305 in FIG. 5 are assumed to be the same lor 
all the packets. That is, it is assumed that the pattern in 
the pattern hash table that matches the packet pattern 
is (*, *, 23) and the hash value of the packet pattern is 
3, i.e., the slot number is 3. 

[0037] First, packet 1 (cip_1 , cpo_1 , 23) is sent by the 
client 1 -1 . Since the VCP list contains nothing (S306, 
FIG. 6), VCP (cipj , cpo_1 , 5-1) is created (S307 : FIG. 
7), and packet 1 is transferred to the server 5-1 (S308) 
identified in slot 3. After that, the server information in 
slot 3 is assumed to have been updated from 5-1 to 5-2 
by the distribution ratio adjusting unit 7. This updating 
will be described later. Next, packet 2 (cip_1 , cpo_1 , 23) 
is sent by the client 1 -1 that sent packet 1 . Packet 2 is 
transferred to the server 5-1 (S309) since the VCP list 
contains VCP (cip_1, cpo_1, 5-1) though the server in- 
formation in slot 3 has been updated to server 5-2 
(S306, FIG. 8). That packet 2 is transferred to the server 
5-1 as opposed to the server 5-2 means that the inter- 
ruption of the service being processed is prevented. 
Next, packet 3 (cip_2, cpo_2, 23) is sent by the client 
1 -2. The VCP list contains no VCP information about the 
client 1-2 (S306, FIG. 8). Thus, VCP (cip_2, cpo_2, 5-2) 
is added to the VCP list (S307, FIG. 9) and packet 3 is 
then transferred to the server 5-2 specified in the slot 3 
(S308). Finally, packet 4 (cipj, cpoj, 23) is sent by 
the client 1-1. This packet 4 is assumed to be the last 
one. Since the VCP list contains VCP (cipj, cpo_1, 
5-1 ) (S306, FIG. 9), packet 4 is transferred to the server 
5-1 (S309). Since this packet is the last one (S31 0), VCP 
(cip_1 , cpoj , 5-1) is deleted from the VCP list (S311, 
FIG. 10). 

[0038] As described above, VCP information is added 
to the slot VCP list at the commencement of service and 
maintained until the service is terminated. At the time of 



access to a slot, the VCP list the slot has is examined. 
If an incoming packet is one for a service in process, 
then it will be delivered to a server already specified in 
the VCP list. Therefore, it becomes possible to prevent 
5 an interruption of the service in process even when a 
change is made to server information that a slot holds 
at the service distribution ratio control time. 
[0039] The more slots the server is assigned, the 
more services it is assigned. Thus, the service distribu- 
te tion ratio can be controlled by the ratio of the numbers 
of slots assigned to the servers. When a server is over 
loaded, the average connection time becomes long and 
the number of connections decreases, which is em- 
ployed as the criterion for judging whether a server is 
15 over loaded. It therefore becomes possible to circum- 
vent a misjudgment of over loading as made when the 
number of connections to a server increases temporarily 
or connection itself is little made to it. 
[0040] A third embodiment of the present invention is 
20 directed to a network load balancing device which is 
characterized in that the distribution ratio adjusting unit 
judges the level of the load on each second control unit 
on the basis of the average time of connection of data 
to it and the number of connections of data to it and bal- 
2S ances the load on each second control unit by adjusting 
the ratio of distribution of data to the second control 
units. 

[0041] How to converge the distribution ratio to the 
servers dynamically to the optimum value, in other 

30 words, how to change the conversion table 4-a so that 
the distribution ratio to the servers will be converged dy- 
namically to the optimum value according to the load 
conditions and the failure conditions of the servers will 
be described below. 

35 [0042] FIG. 11 is a flowchart for measurement of the 
load on each server. 

[0043] In step S111, the load measurement unit 6 
measures the load on each of the servers. The load 
measurement is made based on measured values at the 

40 packet transmission time. Thus, since there is no need 
for communications and server's processing (load 
measurement and notification), the inventive device can 
be installed and operated without affecting the servers. 
For load measurement, the following set of operations 

45 is repeated. That is, the set of operations comprises re- 
setting the previous measurements, waiting for n sec- 
onds, acquiring measurements (the total of connection 
time and the number of connections), and notifying the 
distribution ratio adjusting unit 7 of the measurements 

50 acquired. As the load on each server, use is made of the 
connection average time and the number of connections 
per unit time, which are obtained from the measured 
time interval between the moment that a connect re- 
quest signal is sent to the server and the moment that 

ss a connect completion signal is received from the server 
prior to the transmission of packet data. 
[0044] In step S1 1 2, the load measurement unit 6 no- 
tifies the distribution ratb adjusting unit 7 of those re- 
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suits. 

[0045] FIGs. 12 and 13 are a flowchart for load bal- 
ancing. 

[0046] In step S911, the distribution ratio adjusting 
unit 7 receives notification of load conditions from the s 
load measurement unit 6 and then performs load bal- 
ancing described below. 

[0047] In step S91 2, the initial value of a variable i is 
set to 1 in order to perform load balancing starting with 
the first server. In step S913, a decision is made as to 
whether the first server is over loaded or stopped. In the 
example of the above-described load measurement, a 
server for which the connection average time per unit 
time is longer than a reference connection average time 
and the number of connections per unit time is less than 
a reference number of connections is considered to be 
under over load (FIGs. 1 5A and 1 5B), which is illustrated 
in a flowchart of FIG. 14. 

[0048] In FIG. 14, NS is the number of servers, cti is 
the connection average time for server i, CTi is the ref- 
erence connection average time for server i, cni is the 
number o1 connections for server i, CNi is the reference 
number of connections for server i, and OLi is the over 
load flag for server i which is set ON when it is over load- 
ed. FIG. 15A shows a connection average time versus 
load characteristic of a server, which illustrates that the 
connection average time increases because the con- 
nection time contains a long time-out period due to con- 
nection failures when connection processing cannot 
overtake connection requests because of over load con- 
ditions and thus the number of connection failures in- 
creases. The connection average time is obtained from 
the total connection time and the connection time. FIG. 
1 5 shows a number of connections versus load charac- 
teristic, which illustrates that the number of connections 
per unit time decreases as the connection time becomes 
longer. The over load judgment is made on the basis of 
the two above-described features which can be ob- 
served when a server enters the over load state. The 
reason why two indexes of the connection average time 
and the number of connections are used is to prevent 
misjudgment based on the following two points. The first 
point is that, since the number of connect requests the 
servers can accept simultaneously is limited, when 
more connect requests than the server can accept oc- 
curs, the connection average time becomes long even 
if the servers are not in the over load state. This means 
that the number of connections the severs can accept 
simultaneously is a bottleneck. The number of connec- 
tions becomes a maximum value or so. The other point 
is that, when the number of service requests (connect, 
requests) to servers reduces, the number of connec- 
tions also reduces proportionally. That is, the load on 
the servers decreases because of a small number of re- 
quests. Therefore, the use of both the connection aver- 
age time and the number of connections permits the 
load judgment to be made correctly. 
[0049] The connection average time reference value 
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and the number-of -connections reference value used 
here are assumed to be the minimum connection aver- 
age time per unit time and the maximum number of con- 
nections per unit time, respectively, which have been 
obtained so far. This is based on the fact that for a server 
which is in the over load state the connection average 
time increases and the number of connections decreas- 
es. By the use of both the connection average time and 
the number of connections for load judgment, it be- 
comes possible to circumvent misjudgment that might 
occur if the number of connections to servers exceeded 
temporarily the number the servers can accept simulta- 
neously or the number of connect requests by clients 
decreased. 

[0050] In step S914, for the server considered to be 
over loaded, the number of slots set in the conversion 
table 4-a for that server is set as the maximum number 
of slots (FIGs. 16 and 17). In FIG. 17, OLi is the over 
load flag for server i, OLHi is the over-load experience 
flag for server i, SLTi is the number of slots for server i, 
MAXi is an upper limit on the number of slots for server 
i, MINi is a lower limit on the number of slots for server 
i, and n is a quantity of adjustment for the number of 
slots. 

[0051] For a server considered to be over loaded, the 
number of slots set in the pattern hash table in the con- 
version table 4-a for that server is decreased by a dec- 
rement adjustment quantity in step S917. The decre- 
ment adjustment quantity, indicating the number of sbts 
to be decreased, has been determined in step S91 5. 
The decrement adjustment quantity, which is an integer 
of, for example, three, must be set so as not to become 
smaller than the minimum number of slots allocated for 
that server. After the decrement adjustment quantity has 
been determined, a decision is made in step S916 as to 
whether or not it is equal to zero. If it is not, the number 
of slots for that server set in the hash table in the con- 
version table 4-a is decreased by the decrement adjust- 
ment quantity in step S917. In step S918, the over-load 
experience flag for that server having its allocated slots 
decreased is set ON. 

[0052] Next, in step S919, a decision is made as to 
whether there are other servers which are not over load- 
ed nor stopped. The criterion for this decision is the 
same as that in step S91 3. If there is no other server 
which is not over loaded nor stopped, then the proce- 
dure goes to step S927 to be described later. If the de- 
cision in step S919 is YES, a decision is made in step 
S920 as to whether or not, of the servers detected in 
step S919, there are servers for which the number of 
slots can be increased. If there are, a decision is made 
in step S921 as to whether or not, of the servers detect- 
ed in step S920, there are servers for which the experi- 
ence flag is OFF. If there are, then an arbitrary server, 
for example, a server which is the most lightly loaded, 
is selected from the servers in step S922. If there is no 
server for which the experience flag is OFF, then the pro- 
cedure goes to step S923 in which an arbitrary server, 
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for example, a server which is the most lightly loaded, 
is selected from the servers detected in step S920. In 
step S924, the number of slots which is set in the con- 
version table 4-a for that server selected in step S922 
or S923 is set as the minimum number of slots (FIG. 1 6). 
Next, in step S926, the number of slots set in the pattern 
hash table in the conversion table 4-a for that server is 
increased by an increment adjustment quantity, which 
is determined in step S925. The increment adjustment 
quantity, which corresponds to a decrease in the 
number of slots for the other server considered to be 
over loaded, must be set so as not to become larger 
than the maximum number of slots set for that other 
server. After the increment adjustment quantity has 
been determined, the number of slots set in the pattern 
hash table in the conversion table for that server which 
is the most lightly loaded is increased by the adjustment 
quantity in step S926. In step S930, a decision is made 
as to whether all the servers have been handled. 
[0053] If the decision in step S919 is NO and the de- 
cision in step S920 is NO, then the procedure goes to 
step S927 in which the number of slots, which has been 
decreased in step S917, is restored to the original 
number. And then, a decision is made in step S930 as 
to whether all the servers have been handled. 
[0054] If, in step S91 9, the measurement of a server 
by the load measurement unit 6 shows an abnormal val- 
ue, it is decided that the server is stopped. After that, in 
step S928, a decision is made as to whether all other 
servers are stopped. If not all servers are stopped : then 
the number of slots for that server considered to be 
stopped is made zero in step S929. The procedure then 
goes to step S919 in which a decision is made as to 
whether there is any other server which is not over load- 
ed nor stopped. 

[0055] FIG. 1 8 is a flowchart for deciding that a server 
is in the stopped state because of abnormality due to it 
being too much over loaded. In this figure, NS is the 
number of servers, Ci is a number-of -connections refer- 
ence value for server i, and 0 (Ci) is a reference value 
optimization function. By making the number of slots for 
a server considered to be in the stopped state zero, 
service distribution to the server can be stopped and the 
server stopping can be hidden from clients. 
[0056] FIG. 1 9 shows variations in the number of slots 
for servers. Server 1 is one which is considered to be 
over loaded and for which the experience flag is ON. 
The number of slots is decreased by n. Server 2 is one 
which is considered to be over loaded and for which the 
experience flag is OFF. The experience flag is set ON 
and the number of slots is decreased by m. Server 3 is 
one which, although its experience flag is ON, is not con- 
sidered to be over loaded. Server 4 is one which is not 
considered to be over loaded and for which the experi- 
ence flag is OFF. Thus, the number of slots for server 4 
is incremented by n + m where n and m are decrements 
in the number of slots for servers 1 and 2, respectively. 
[0057] If, in step S913, it is decided that a server is 



not over loaded nor stopped, then the procedure goes 
to step S930 in which a decision is made as to whether 
or not all the servers have been handled. If all the serv- 
ers have not been handled, the variable i is incremented 
s in step S931 . The procedure then returns to step S91 3 
to handle the next server. 

[0058] Next, in step S932, a decision is made as to 
whether or not the over-load experience flag is ON for 
all the servers. In the presence of at least one server for 
io which the over-load flag is not ON, return is made to 
step S911 to receive the load measurements. If the ex- 
perience flag is ON for all the servers, it is returned to 
OFF for all the servers. In step S935, the decrement ad- 
justment value, the initial value of which has been set to 
three in the previously described example, is changed 
to a proper value, for example, two, which is obtained 
by decrementing the initial value by one. The procedure 
then returns to step S911 to receive the load measure- 
ments. 

[0059] That, in step S932, the over-load experience 
flag is ON for all the servers means that all the servers 
have undergone an adjustment of the number of slots. 
Further, the adjustment width becomes decreased by 
decreasing the decrement adjustment quantity In step 
S935. This means that the adjustment gradually con- 
verges in a limited number of times. The load on each 
server can be made to approach an optimum value. 
[0060] The more slots the server is assigned, the 
more services it receives. Thus, the service distribution 
ratio can be controlled by the ratio of the numbers of 
slots assigned to the servers. When a server is over 
loaded, the connection average time increases and the 
number of connections to servers decreases, which is 
employed to detect the over-load state of the server. It 
therefore becomes possible to circumvent a misjudg- 
ment of over load as made when the number of connec- 
tions to servers increases temporarily or connection it- 
self is little made. 

[0061] A fourth embodiment of the present invention 
is directed to a network load balancing device which is 
arranged as follows: 

[0062] The conversion information storage unit in- 
cludes a table in which a correspondence relationship 
of the first control unit to the second control units is es- 
tablished and a unit that maintains the correspondence 
relationship between the first control unit and the second 
control units throughout transmission of a sequence of 
packets. The distribution ratio adjusting unit comprises 
a unit that, when the measurement of the load on each 
second control unit shows an abnormal value, judges 
that it is in the stopped state, a unit that compares the 
connection average time and the number of connections 
for the second control unit with the connection average 
time reference value and the numberof-connections 
reference value which are the minimum connection av- 
erage time and the maximum number of connections for 
the second control units, respectively, and considers a 
second control unit as over loaded for which the con- 
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nection average time is sufficiently larger than the cor- 
responding reference value and the number of connec- 
tions is sufficiently smaller than the corresponding ref- 
erence value, a unit that manages the service distribu- 
tion ratio to the second control units as numbers allocat- 
ed to the second control units. This management unit 
comprises a stopped state management unit that, in the 
presence of a second control unit considered to be in 
the stopped state, distributes the allocated number of 
that second control unit to second control units which 
are not in the stopped state at the ratio of the allocated 
numbers of those second control units, and a over load 
management unit that decreases the allocated number 
of a second control unit considered to be over loaded 
and increases the allocated numbers of second control 
units which are not over loaded. This over load manage- 
ment unit includes a unit that decreases the incremental 
or decremental value of the allocated numbers of the 
second control units with each adjustment cycle to 
thereby converge the service distribution ratio. The load 
measuring unit includes a unit that periodically meas- 
ures the connection average time and the number of 
connections per unit time for each second control unit 
on the basis of data relayed to it and notifies the distri- 
bution ratio adjusting unit of the statuses of the second 
control units. 

[0063] The more slots the server is assigned, the 
more services it receives. Thus, the service distribution 
ratio can be controlled by the ratio of the numbers of 
slots assigned to the servers. When a server is over 
loaded, the connection average time increases and the 
number of connections to the server decreases, which 
is employed to detect the over-load state of each server. 
It therefore becomes possible to circumvent a misjudg- 
ment of over load as made when the number of connec- 
tions to a server increases temporarily or connection it- 
self is little made. 

[0064] A fifth embodiment of the present invention is 
directed to a network load balancing device which fur- 
ther includes a status management unit which pre- 
serves the status before adjustment and restores the 
status to the original status at recovery time. 
[0065] The more slots the server is assigned, the 
more services it receives. Thus, the service distribution 
ratio can be controlled by the ratio of the numbers of 
slots assigned to the servers. When a server is over 
loaded, the connection average time increases and the 
number of connections to the server decreases, which 
is employed to detect the over-load state of each server. 
It therefore becomes possible to circumvent a misjudg- 
ment of over load as made when the number of connec- 
tions to a server increases temporarily or connection it- 
self is little made. 

[0066] A sixth embodiment of the present invention is 
directed to a network load balancing device which is ar- 
ranged as follows: 

[0067] The conversion information storage unit in- 
cludes a table in which a correspondence relationship 
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of the first control unit to the second control units is es- 
tablished and a unit that maintains the correspondence 
relationship of the first control unit to the second control 
units throughout transmission of a sequence of data 
5 packets. The distribution ratio adjusting unit comprises 
a unit that, when the measurement of the load on each 
second control unit shows an abnormal value, judges 
that it is in the stopped state, and a stopped state man- 
agement unit that manages the distribution ratio to the 
10 second control units as the allocated numbers for the 
second control units in the conversion information and, 
in the presence of a second control unit considered to 
be in the stopped state, distributes the allocated number 
of that second unit to second control units which are not 
in the stopped state at the ratio of the allocated numbers 
of those second control units. 

[0068] The more slots the server is assigned, the 
more services it receives. Thus, the service distribution 
ratio can be controlled by the ratio among the numbers 
of slots assigned to the servers. When a server is over 
loaded, the connection average time increases and the 
number of connections to the server decreases, which 
is employed to detect over loading of the server. It there- 
fore becomes possible to circumvent a misjudgment of 
over load as made when the number of connections to 
a server increases temporarily or connection itself is lit- 
tle made. 

[0069] Although the flow of load measurement and 
the flow of load balancing have been described in terms 
of several steps, other examples of these steps will be 
described next. 

[0070] A seventh embodiment of the present inven- 
tion is directed to a network load balancing device which 
is arranged as follows: 

[0071] The load measuring unit includes a unit that 
measures the number of connection errors and the 
number of connections per unit time for each second 
unit on the basis of data relayed to it and notifies the 
distribution ratio adjustment unit of the measurements 
at regular intervals. The distribution ratio adjusting unit 
further comprises a unit that, when the measurement of 
the load on each second control unit shows an abnormal 
value, judges that it is in the stopped state, compares 
the connection error rate and the number of connections 
for each second control unit with a connection error rate 
reference value and a number-of -connections reference 
value which are the minimum connection error rate, 
which is represented by the ratio of the number of con- 
nection errors to the number of connections, and the 
maximum number of connections for the second control 
units, respectively, and considers a second control unit 
as over loaded for which the connection error rate is suf- 
ficiently larger than the corresponding reference value 
and the number of connections is sufficiently smaller 
than the corresponding reference value. 
[0072] In step S1 1 1 of FIG. 1 1 , the load measurement 
unit 6 measures the time interval between the time of 
transmission of a connect request signal to a server and 
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the time of reception of a connect complete signal from 
it prior to transmission of packet data and uses, as the 
measurements of the load on it, the connection average 
time and the number of connections per unit time which 
are sought from the measured time interval. As an al- 
ternative, it may be arranged to detect connection errors 
for the connect request signal and use the connection 
error rate (the ratio of the number of connection errors 
detected to the number of connections) and the number 
of connections per unit time. In this case, the criterion 
for considering a server to be over loaded in step S91 3 
is that the connection error rate per unit time is higher 
than a connection error rate reference value and the 
number of connections per unit time is smaller than a 
number-of -connections reference value. The connec- 
tion error rate reference value and the number-of-con- 
nections reference value used here are respectively de- 
fined to be the minimum correction error rate and the 
maximum number of connections per unit time, which 
have been obtained so far. 

[0073] FIG. 20 is a flowchart for high bad judgment, 
which employs the connection error rate per unit time 
and the number of connections per unit time as the cri- 
terion in step S913 of FIG. 12 for deciding whether a 
server is over loaded. In this figure, NS is the number of 
servers, cei is the connection average time for server i, 
CEi is the connection average time reference value for 
server i, cni is the number of connections for server i, 
CNi is the number-of -connect ions reference value for 
server i, and OLi is the high load flag for server i. 
[0074] Since the number of connection errors and the 
number of connections are measured, load balancing 
can be made even if another route is taken between a 
server and a client. 

[0075] An eighth embodiment of the present invention 
is directed to a network load balancing device which is 
characterized in that the load measuring unit further in- 
cludes a unit that measures the number of virtual circuits 
(VCs) for each of the second control units on the basis 
of data relayed to it and notifies the distribution adjust- 
ment unit of the measurement at regular intervals, and 
the distribution ratio adjustment unit further includes a 
unit that considers a second control unit for which the 
number of VCs exceeds a preset reference value to be 
over loaded. 

[0076] Since the number of VCs is used for over load 
judgment, it is possible to estimate the load on a server 
even in a case where the time interval between the com- 
mencement and termination of a service is long and 
hence the number of connections is small. 
[0077] A ninth embodiment of the present invention is 
directed to a network load balancing device which is 
characterized in that the load measurement unit meas- 
ures the response time between the commencement 
and termination of transmission for each of the second 
control units on the basis of data relayed to it and notifies 
the distribution ratio adjustment unit of the result at reg- 
ular intervals, and the distribution ratio adjustment unit 
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further includes a unit that considers a second control 
unit to be over loaded for which the response time ex- 
ceeds a preset reference value. 
[0078] Alternatively, in step Sill of FIG. 11, the load 
5 measurement unit 6 may measure the number of VCs 
for each second server and provide the number of VCs 
per unit time as the measurement of the load on it. In 
this case, the criterion in step S91 3 for considering the 
second sever to be over loaded is that the number of 
VCs per unit time is larger than a reference value for the 
number of VCs. The reference value used here is one 
that has been set in advance. 

[0079] FIG. 21 is a flowchart for over-load judgment, 
which employs the number of VCs per unit time as the 
criterion in step S913 of FIG. 12 for deciding whether a 
server is over loaded. In this figure, NS is the number of 
servers, Pi is the number of VCs, the response average 
time, and the imitated service response average time 
for server i, Ci is the user-set reference value for server 
i, and OLi is the over load flag for server i. 
[0080] Since the service response time is employed 
for over-load judgment, the real-time load judgment can 
be made. 

[0081] A tenth embodiment of the present invention is 
directed to a network load balancing device which is 
characterized in that the load measurement unit per- 
forms an operation determined by imitating the first con- 
trol unit in data sent by each second control unit, meas- 
ures the service time required to perform the operation, 
and notifies the distribution ratio adjustment unit of the 
average time, and the distribution ratio adjustment unit 
further includes a unit that decides that a second control 
unit is over loaded for which the service average time is 
larger than a preset reference value. 
[0082] Alternatively, in step S111 of FIG. 11, the load 
measurement unit 6 may measure the response time 
between the commencement and termination of a serv- 
ice for each server and provide the response average 
time per unit time as the measurement of the load on it. 
In this case, the criterion in step S91 3 of FIG. 1 2 for con- 
sidering the sever to be over loaded is that the response 
average time per unit time is larger than a response av- 
erage time reference value. The reference value used 
here is one that has been set in advance. 
[0083] In FIG. 21, the response average time (11-b) 
is employed as the criterion in step S913 of FIG. 1 2 for 
deciding whether a server is over loaded. 
[0084] Since the response time for an imitated service 
is used for over load judgment, load measurement that 
reflects a difference in characteristic between each 
service can be made. 

[0085] Further, in step S111 of FIG. 11, as a dummy 
client the load measurement unit 6 may perform some 
service operation in imitation of a client, measures the 
response time between the commencement and termi- 
nation of the service, and provides the imitative service 
response average time as the load measurement, as 
shown in FIG. 22. In this case, the criterion in step S91 3 
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of FIG. 12 for considering the sever to be over loaded 
is that the imitative service response average time per 
unit time is larger than an imitative service response av- 
erage time reference value. The reference value used 
here is one that has been set in advance. 
[0086] In FIG. 21, the imitative service response av- 
erage time (11 -c) is employed as the criterion in step 
S913 of FIG. 12 for deciding whether a server is over 
loaded. 

[0087] An eleventh embodiment of the present inven- 
tion is directed to a network load balancing device which 
is characterized by further comprising a unit that allows 
an operator to set the reference value used in the dis- 
tribution ratio adjustment unit for over load judgment. 
[0088] As described previously, In step S913 of FIG. 
1 2, the minimum connection average time per unit time, 
the maximum number of connections per unit time, or 
the connection error rate reference value per unit time, 
which are obtained by measurements, is used as the 
connection average time reference value, the number- 
of-connections reference value., or the connection error 
rate reference value. However, the inventive device may 
be modified such that an operator can specify these ref- 
erence values. 

[0089] FIG. 23 is a flowchart for over load judgment 
in which the connection average time reference value 
and the number-of- connections reference value are set 
by the operator. In this figure, NS is the number of serv- 
ers, cti is the connection average time for server i, CTi 
is the connection average time reference value for serv- 
er i, cni is the number of connections for server i, CNi is 
the number-of -connections reference value for server i, 
and OLi is the over load flag for server i. 
[0090] FIG. 24 is a flowchart for over load judgment 
in which the connection error rate reference value and 
the number-of -connect ions reference value are set by 
the operator. In this figure, NS is the number of servers, 
cei is the connection average time for server i, CEi is the 
connection average time reference value for server i, cni 
is the number of connections for server i, CNi is the 
number-of -connections reference value for server i, and 
OLi is the over- load flag for server i. 
[0091] If, since the operator sets the reference values, 
optimum values are known, the settings will immediately 
be reflected in the load measurement. 
[0092] A twelfth embodiment of the present invention 
is directed to a network load measurement device which 
is characterized in that the distribution ratio adjustment 
unit further comprises a unit that detects a second con- 
trol unit considered to be stopped, preserves the refer- 
ence values at that point, and employs the preserved 
reference values after recovery of the stopped server. 
[0093] The reference value, such as the connection 
average reference value, the number-of -connections 
reference value, the connection error rate reference val- 
ue, the number-of-VCs reference value, the response 
average time reference value, or the imitative service 
response average time reference value, may be pre- 



served in the status management unit 10 on a timely 
basis. For example, the reference value may be pre- 
served at the time a stopped server is detected. In this 
case, wtien the stopped server is recovered, the pre- 
s served reference value is fetched and used. 

[0094] Since the reference values when a stopped 
server is detected are preserved, the service will not be 
stopped again under the same situation. 
[0095] A thirteenth embodiment of the present inven- 
tion is directed to a network load measurement device 
which is characterized in that the distribution ratio ad- 
justment unit further comprises a unit that determines 
an increment or decrement value for the number of slots 
allocated to a second control unit when it is considered 
to be over loaded according to the number of slots which 
has already been allocated to it at that time. 
[0096] In step S91 5 of FIG. 12, a decrement value to 
be determined is selected to be a preset integer of, for 
example, three. Alternatively, a predetermined percent- 
age of the number of slots set in the conversion table 
4-a for a server which is considered to be over loaded 
may be set as the decrement value. 
[0097] Since the quantity of adjustment is increment- 
ed or decremented by a percentage of the number of 
slots allocated to a server, the quantity of adjustment 
will not vary greatly from server to server. 
[0098] A fourteenth embodiment of the present inven- 
tion is directed to a network load balancing device which 
is characterized in that the distribution ratio adjustment 
unit further includes a unit that distributes the number 
of slots assigned to a second control unit when it is con- 
sidered to be over loaded to all other second control 
units which are not over loaded according to the ratio of 
the numbers of slots assigned to those second control 
units at that time. 

[0099] The decrement value for the number of sbts 
assigned to a server which is considered to be over load- 
ed is determined in step S9 1 5 of FIG . 1 2. The decrement 
value may be updated by seeking the average value of 
load information and then the numbers of slots allocated 
for servers whose differences from the average value 
are greater than a predetermined value according to the 
ratio among their distances from the average value. 
[0100] FIG. 25 is a flowchart for load balancing oper- 
ation to allow the server capabilities viewed from clients 
to be made constant by seeking servers whose differ- 
ences are a predetermined value greater than the aver- 
age measured value. In this figure, tx is the measured 
value for server x. 

[0101] Since the adjustment quantity is incremented 
or decremented on the basis of the ratio among the 
number of slots allocated for servers which are not over 
loaded, the adjustment quantity will not vary greatly from 
server to server when there is a large difference in 
processing capabilities among the servers. 
[0102] A fifteenth embodiment of the present inven- 
tion is directed to a network load measurement device 
which is characterized in that the distribution ratio ad- 
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justment unit further comprises a unit that seeks the av- 
erage of the measured values of the loads on second 
control units and then determines the numbers of slots 
allocated for second control units whose measured val- 
ue differ from the average by width 8 according to the 
ratio among their distances from the average. 
[01 03] The incremental quantity of adjustment for the 
number of slots for a server considered to be not over 
loaded nor stopped is determined in step S915 of FIG. 
12. The increment quantity may be updated by seeking 
the numbers of slots allocated for servers whose differ- 
ences from the average value of load information are 
greater than a predetermined value according to the ra- 
tio among their distances from the average value. 
[01 04] Since the server response delays viewed from 
clients are made constant from client to client, each cli- 
ent can be provided with services equally. 
[0105] In step S915 of FIG. 12, the incremental ad- 
justment quantity for each server which is considered to 
be not over loaded or stopped may be set equal to the 
decremental adjustment quantity for another server 
considered to be over loaded (FIGs. 17 and 18). 
[0106] In step S915 of FIG. 12, the incremental ad- 
justment quantities for servers considered to be not over 
loaded or stopped may be determined according to the 
numbers of slots allocated for those servers in the con- 
version table 4-a as shown in FIG. 26. 
[0107] FIG. 27 is a flowchart for allocating slots for 
servers according to the number of slots allocated for 
them in the conversion table 4-a, which is used in place 
of stepS11 of FIG. 18. 

[0108] A sixteenth embodiment of the present inven- 
tion is directed to a network load balancing device which 
is characterized by further comprising a failure monitor 
unit that performs a ping command to each second unit 
at regular intervals, decides that a second unit which 
causes a certain number of time-outs is in the stopped 
state, and notifies the distribution ratb adjustment unit 
of it. 

[0109] In step S91 3 of FIG. 12, a server for which the 
measurement by the load measurement unit 6 indicates 
an abnormal value is considered to be stopped. A failure 
monitor unit 8 that performs a ping command to each 
second unit at regular intervals, decides that a second 
unit which causes a certain number of time-outs is in the 
stopped state, and notifies the distribution ratio adjust- 
ment unit of it may be provided separately as shown in 
FIG. 28. Of course, the server stop judgment by the load 
measurement unit 6 and the server stop judgment by 
the failure monitor unit 8 may be used in combination. 
[0110] Thereby, the accuracy of the detection of serv- 
er stops is increased. 

[0111] A seventeenth embodiment of the present in- 
vention is directed to a network load balancing device 
which is characterized in that the failure monitor unit fur- 
ther includes a unit that tries whether it is possible to 
open a service port to transmit information from each 
second control unit, deciding, after a certain number of 



failures, that the transmit information is being stopped, 
and notifies the distribution ratio adjustment unit. 
[0112] As shown in FIG. 28, the failure monitor unit 8 
may be arranged to, in addition to a ping command, try 
5 whether it is possible to open a service port to a server 
service and decide, after a certain number of failures, 
that the service is in the stopped state. 
[011 3] Since stops on the application layer can be de- 
tected, the hiding and maintenance of failures can be 
io performed on a service-by-service basis. 

[0114] The present invention has been described so 
far in terms of one form, that the network service server 
load balancing device 3 receives packets from the cli- 
ents 1-1, 1-2,.., 1-m over the network 2, then transfers 
them to the servers 5-1, 5-2,.., 5-n, and receives serv- 
ices from the servers, then return them to the clients 
over the network. The network service server load bal- 
ancing device 3 may be employed in other forms: it is 
placed on one of the servers as shown in FIG. 29A; the 
device is used to send packets from the clients to serv- 
ers, but not used to return services from the servers to 
the clients as shown in FIG. 29B; and the device also 
accommodates another network as an object of load 
balancing as shown in FIG. 29C. 
[0115] An eighteenth embodiment of the present in- 
vention is directed to a network load balancing device 
characterized in that one second control unit includes 
the network load balancing unit. 
[011 6] Thus, there is no necessity of addition of a new 
hardware device at the time the system is installed. 
[0117] A nineteenth embodiment of the present inven- 
tion is directed to a network load balancing device which 
further includes another communication path directed 
from the second to the first control unit in addition to the 
communication path that passes through the network 
load balancing device. 

[0118] Thereby, the overall processing efficiency of 
the third control unit can be increased. 
[011 9] A twentieth embodiment of the present inven- 
tion is directed to a network load balancing device char- 
acterized by further comprising a unit consisting of a 
group of the second control units of multisegment con- 
figuration to which another network is connected. 
[0120] Thereby, it becomes possible for the third con- 
trol unit to make load balancing over the entire network. 
The load balancing can be made using the present flow 
of data. The load will not be increased due to load meas- 
urement. 

[0121] A twenty-first embodiment of the present in- 
vention is directed to a network load balancing device 
characterized in that the first control unit is a client com- 
puter connected to a network, the second control units 
are server computers connected to the network, and the 
conversion information storage unit is a conversion ta- 
ble from the first control unit to the second control units. 
[0122] Well-balanced efficient services can be provid- 
ed because the load on each of servers that make up a 
network is measured and packets are sent from a client 
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to servers according to the load condition of each server. 
Further, no server switching occurs in the middle of a 
sequence of services because it is known which server 
a sequence of packets is sent to. 

[0123] A twenty-second embodiment of the present & 
invention is directed to a network load balancing meth- 
od, for use with a system in which data is transferred 
from a first control unit to a plurality of second control 
units over a network, which comprises: a conversion in- 
formation storing step of storing conversion information 
which represents a correspondence relationship of the 
first control unit to the second control units; a distributive 
relay step of transferring the data sent by the first control 
unit to one or more of the second control units according 
to the correspondence relationship in the conversion in- 
formation stored by the conversion information storing 
step; and a distribution ratio adjusting step of adjusting 
the correspondence relationship in the conversion infor- 
mation stored by the conversion information storing 
step; a load measuring step of measuring the load on 
each of the second control units and adjusting the cor- 
respondence relationship in the conversion information 
according to the load conditions of the second control 
units. 

[01 24] Well-balanced efficient services can be provid- 
ed because packets are sent from a client to servers 
according to the load condition of each server. Further, 
no server switching occurs in the middle of a sequence 
of services because it is known which server a sequence 
of packets is being sent to. 

[0125] A twenty-third embodiment of the present in- 
vention is directed to a computer-readable storage me- 
dium for use with a system in which data is transferred 
from a first control unit to a plurality of second control 
units over a network, which is recorded with a load bal- 
ancing program which allows a computer to perform; a 
conversion information storing step of storing conver- 
sion information which represents a correspondence re- 
lationship of the first control unit to the second control 
units; a distributive relay step of transferring the data 
sent by the first control unit to one or more of the second 
control units according to the correspondence relation- 
ship in the conversion information stored by the conver- 
sion information storing step; and a distribution ratio ad- 
justing step of adjusting the correspondence relation- 
ship in the conversion information stored by the conver- 
sion information storing step; a load measuring step of 
measuring the load on each of the second control units 
and adjusting the correspondence relationship in the 
conversion information according to the load conditions 
of the second control units. 

[01 26] Well-balanced efficient services can be provid- 
ed because packets are sent from a client to servers 
according to the load condition of each server. Further, 
no server switching occurs in the middle of a sequence 
of services because it is known which server a sequence 
of packets is being sent to. 

[0127] A twenty-forth embodiment of the present in- 



vention is directed to a network load balancing system 
which comprises a first control unit which sends data, a 
plurality of second control units which receives the data, 
and a third control unit which transfers the data sent by 
the first control unit to the second control units. The third 
control unit comprises: a conversion information storing 
unit for storing conversion information which represents 
a correspondence relationship of the first control unit to 
the second control units; a distributive relay unit for 
transferring the data sent by the first control unit to one 
or more of the second control units according to the cor- 
respondence relationship in the conversion information 
stored in the conversion information storing unit; and a 
distribution ratio adjusting unit for adjusting the corre- 
spondence relationship in the conversion information 
stored in the conversion information storing unit; and a 
load measuring unit for measuring the 'load on each of 
the second control units and notifying the distribution ra- 
tio adjusting unit of load conditions of the second control 
units in order to allow the distribution ratio adjusting unit 
to adjust the correspondence relationship in the conver- 
sion information stored in the conversion information 
storing unit. 

[01 28] Well-balanced efficient services can be provid- 
ed because packets are sent from a client to servers 
according to the load condition of each server. Further, 
no server switching occurs in the middle of a sequence 
of services because it is known which server a sequence 
of packets is being sent to. 

[0129] A twenty-fifth embodiment of the present in- 
vention is directed to an adjustment device for use with 
a system in which data is sent from a first control unit to 
a plurality of second control units, which comprises: a 
conversion information storing unit for storing conver- 
sion information which represents a correspondence re- 
lationship ol the first control unit to the second control 
units; a distributive relay unit for transferring the data 
sent by the first control unit to one or more of the second 
control units according to the correspondence relation- 
ship in the conversion information stored in the conver- 
sion information storing unit; and a distribution ratio ad- 
justing unit for adjusting the correspondence relation- 
ship in the conversion information stored in the conver- 
sion information storing unit; and a load measuring unit 
for measuring the load on each of the second control 
units and notifying the distribution ratio adjusting unit of 
load conditions of the second control units in order to 
allow the distribution ratio adjusting unit to adjust the 
correspondence relationship in the conversion informa- 
tion stored in the conversion information storing unit. 
[01 30] Well-balanced efficient services can be provid- 
ed because packets are sent from a client to servers 
according to the load condition of each server. Further, 
no server switching occurs in the middle of a sequence 
of services because it is known which server a sequence 
of packets is being sent to. 

[0131] A twenty -sixth embodiment of the present in- 
vention is directed to a control device for use with a sys- 
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tern in which data is sent from a first control unit to a 
plurality of second control units, which comprises: a con- 
version information storing unit for storing conversion in- 
formation which represents a correspondence relation- 
ship of the first control unit to the second control units; s 
a distributive relay unit for transferring the data sent by 
the first control unit to one or more of the second control 
units according to the correspondence relationship in 
the conversion information stored in the conversion in- 
formation storing unit. 10 
[0132] No server switching occurs in the middle of a 
sequence of services because it is known which server 
a sequence of packets is being sent to. 
[0133] A twenty-seventh embodiment of the present 
invention is directed to a network load balancing device 15 
for use with a system in which data is sent from a client 
to servers, which comprises a unit which stores relevant 
data so that it will be processed in a corresponding serv- 
er and a unit which decides whether each server is over 
loaded and, if it is, balances the load on that server so 20 
that it will be distributed to the other servers. 
[01 34] Well-balanced efficient services can be provid- 
ed because packets are sent from a client to servers 
according to the load condition of each server. Further, 
no server switching occurs in the middle of a sequence 25 
of services because it is known which server a sequence 
of packets is being sent to. 

[01 35] The present invention can be applied to a sin- 
gle device, a system or integrated device composed of 
multiple units, or a system in which processing is per- 30 
formed via a network, such as a LAN, so long as they 
can perform the function of the present invention. 
[0136] Also, as shown in FIG. 30, the present inven- 
tion can be carried out by loading a recording medium 
(e.g., a ROM ; a hard disk, or a floppy disk) recorded with 35 
program code for implementing the device according to 
each of the embodiments described previously into a 
system or device and causing a computer (a CPU or an 
MPU) in the system or device to read and execute the 
program code stored on the recording medium. 40 
[01 37] In this case, the program code itself read from 
the recording medium implements the novel function of 
the present invention and the recording medium record- 
ed with the program code is therefore an implementation 
of the present invention. 45 
[0138] As a recording medium for the program code 
use may be made of a floppy disk, a hard disk, an optical 
disk, an opto-magnetic disk, a CD-ROM, a CD-R (Com- 
pact Disk Rewritable), magnetic tape, a nonvolatile 
memory card, a ROM, or a type of recording medium so 
which can be recorded through the use of a network (or 
communications line) such as electronic mail or person- 
al computer communications. 

[01 39] The functions of the previously described em- 
bodiments are performed by the computer executing the 55 
read program code. Or, the OS operating on the com- 
puter can perform part or all of actual processes as in- 
structed by the program code to implement the function 



of the present invention. 

[0140] Furthermore, the function of the present inven- 
tion can also be implemented through the use of an ex- 
pansion board inserted into a computer or an expansion 
unit attached to the computer. That is. the program code 
read from the recording medium is written into a memory 
on the expansion board or expansion unit and a CPU 
on the expansion board or unit performs part or all of 
actual processes as instructed by the program code to 
thereby implement the function of the present invention. 
[0141] As described above, in sharing services 
among servers, the present invention can allocate loads 
for severs automatically and efficiently even with varia- 
tions in server configuration or server status. Therefore, 
a client can obtain a quick supply of service and no tem- 
porary service stop will occur. 



Claims 

1. A network load balancing device (3) for use a sys- 
tem in which data is transferred from a first control 
unit (5-1 ) to a plurality of second control units (1-1 , 
1-2, .... 1-m) over a network, said network load bal- 
ancing device (3) comprising: 

conversion information storage means (4-a) for 
storing conversion information representing a 
correspondence relationship of the first control 
unit (5-1 ) to one of the second control units (1 -1 , 
1-2, 1-m); 

distributive relay means (4) for transferring data 
sent by the first control unit (5-1 ) to one or more 
of the second control units (1-1, 1-2, 1-m) 
on the basis of the conversion information 
stored in the conversion information storage 
means (4-a); 

distribution ratio adjusting means (7) for adjust- 
ing the correspondence relationship in the con- 
version information stored in the conversion in- 
formation storage means (4-a); and 
load measuring means (6) for measuring loads 
on one of the second control units (1-1,1-2,..., 
1 -m) and notifying the distribution ratio adjust- 
ing means (7) of load conditions of one of the 

second control units (1-1,1-2 1 -m) tothere- 

by allow the distribution ratio adjusting means 
(7) to adjust the correspondence relationship in 
the conversion information stored in the con- 
version information storage means (4-a). 

2. The device according to claim 1, wherein the load 
measuring means (6) measures load on each of the 
second control units (1 -1 , 1 -2, .... 1 -m) on the basis 
of an average time of connecting the data to it per 
unit time and the number of connections of the data 
to it per unit time. 
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3. The device according to claim 2, wherein the distri- 
bution ratio adjusting means (7) judges whether 
each of the second control units (1-1, 1-2, .... 1-m) 
is over loaded on the basis of an average time of 
connecting and the number of connections of data 5 
to it and adjusts a ratio of distribution of the data to 
the second control units (1 -1 , 1-2, .... 1-m) to there- 
by balance the load on each of the second control 
units (1*1, 1-2, 1-m). 

io 

4. The device according to claim 1 , wherein the con- 
version information storage means (4-a) includes a 
table in which the correspondence relationship of 
the first control unit (5-1 ) to the second control units 
(1-1, 1-2,..., 1-m) is established and means for is 
maintaining the correspondence relationship of the 
first control unit (5-1) to one of the second control 
units (1-1, 1-2 1-m) throughout transmission of 

a sequence of data, the distribution ratio adjusting 
means (7) comprises means for, when the meas- 20 
urement of the load on each server shows an ab- 
normal value, judging that it is in the stopped state, 
means for comparing the connection average time 
and the number of connections for each second 

control unit (1 -1 , 1-2 1-m) with a connection av- 2S 

erage time reference value and a number-of-con- 
nections reference value which are the minimum 
connection average time and the maximum number 
of connections for one of the second control units 

(1-1, 1-2 1-m), respectively, and judging that 30 

one of the second control units (1-1, 1-2 .... 1-m) is 
over loaded for which the connection average time 
is sufficiently larger than the corresponding refer- 
ence value and the number of connections is suffi- 
ciently smaller than the corresponding reference 35 
value, means (10) for managing the service distri- 
bution ratio to one of the second control units (1 -1 , 
1-2 1-m) as numbers allocated to one of the sec- 
ond control units (1-1, 1-2 ..., 1-m), the manage- 
ment unit (10) including stopped state management 40 
means for, in the presence of the second control 
units (1-1, 1-2, 1-m) considered to be in the 
stopped state, distributing the allocated number of 
the second control units (1-1, 1-2..., 1-m) to the 
second control units (1-1, 1-2, .... 1-m) which are 45 
not in the stopped state at the ratio of the allocated 

numbers of the second control units (1-1, 1-2 

1-m) and over load management means for de- 
creasing the allocated number of the second control 
units (1-1 , 1-2 .... 1-m) considered to be over loaded so 
and increasing the allocated numbers of the second 
control units (1-1, 1-2, 1-m) which are not over 
loaded and the over load management means (6) 
including means for decreasing the incremental or 
decremental value of the allocated numbers of the ss 
second control units (1-1, 1-2, .... 1-m) with each 
adjustment cycle to thereby converge the distribu- 
tion ratio for the second control units (1-1, 1-2, .... 



1-m), and the load measurment means (6) includes 
means for periodically measuring the connection 
average time and the number of connections per 
unit time for each of the second units (1-1, 1-2 .... 
1 -m) on the basis of data relayed to it and notifying 
the distribution ratio adjusting means (7) of the sta- 
tuses of the second control units (1-1, 1-2..., 1-m). 

5. The device according to claim 4, further comprising 
status management means (10) which preserves 
the status before adjustment and restores the sta- 
tus to the original status at recovery time. 

6. The device according to claim 4 or 5, wherein the 
load measuring means (6) includes means for 
measuring the number of connection errors and the 
number of connections per unit time for each of the 
second control un its ( 1 -1 , 1 -2 ... , 1 -m) on the basis 
of data relayed to it and notifies the distribution ratio 
adjustment unit (7) of the measurements at regular 
intervals, and the distribution ratio adjusting means 
(7) further includes means for, when the measure- 
ment of the load on each server shows an abnormal 
value, judges that it is in the stopped state, com- 
pares the connection error rate and the number of 
connections for each of the second control units 
(1 -1 , 1 -2 1 -m) with a connection error rate refer- 
ence value and a number-of-connections reference 
value which are the minimum connection error rate, 
which is represented by the ratio of the number of 
connection errors to the number of connections, 
and the maximum number of connections for the 
second control units (1-1, 1-2, .... 1-m), respective- 
ly and considers a second control unit as over load- 
ed for which the connection error rate is sufficiently 
larger than the corresponding reference value and 
the number of connections is sufficiently smaller 
than the corresponding reference value. 

7. The device according to claim 6, further comprising 
means for allowing an operator to set the reference 
value for over load judgment in the distribution ratio 
adjusting means (7). 

8. The device according to claim 7, wherein the distri- 
bution ratio adjustment means (7) further includes 
means for detecting second control units consid- 
ered to be stopped, preserves reference values at 
that point, and employs the preserved reference 
values after recovery of it. 

9. The device according to claim 8, wherein the distri- 
bution ratio adjustment means (7) further includes 
means for determining an increment or decrement 
value for the number of slots allocated to a second 
control unit when it is considered to be over loaded 
according to the number of slots which has already 
been allocated to it at that time. 
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10. The device according to claim 9, wherein the distri- 
bution ratio adjustment means (7) further includes 
means for distributing the number of slots assigned 
to the second control units (1-1 , 1-2,..., 1-m) when 
it is considered to be over loaded to all other second 
control units (1-1, 1-2...., 1-m) which are not over 
loaded according to the ratio of the numbers of slots 
assigned to the second control units (1-1, 1-2..., 
1 -m) at that time. 

11. The device according to claim 1 , wherein the con- 
version information storage unit (4-a) includes a ta- 
ble in which a correspondence relationship of the 
first control unit (5-1) to the second control units 
(1-1, 1-2..., 1-m) is established and means for 
maintaining the correspondence relationship 
throughout transmission of a sequence of data, and 
the distribution ratio adjusting means (7) comprises 
means for, when the measurement of the load on 
each server shows an abnormal value, judges that 
it is in the stopped state, and stopped state man- 
agement means (10) for managing the distribution 
ratio to the second control units (1-1, 1-2, .... 1-m) 
as allocated numbers for the second control units 

(1-1,1-2 1 -m) in the conversion table and, in the 

presence of one of second control units (1-1, 
1-2, 1-m) considered to be in the stopped state, 
distributing the allocated number of one of the sec- 
ond control units (1-1, 1-2..., 1-m) to the second 
control units (1-1, 1-2, .... 1-m) which are not in the 
stopped state at the ratio of the allocated numbers 
of those second control units (1-1, 1-2 1-m). 

12. The device according to any preceding claim, 
wherein the load measuring means (6) further in- 
cludes means for measuring the number of virtual 
circuits per unit time for each of the second control 
units (1 -1 , 1-2, 1 -m) on the basis of data relayed 
to it and notifies the distribution adjusting ratio 
means (7) of the measurement at regular intervals, 
and the distribution ratio adjustment means (7) fur- 
ther includes means for considers one of the sec- 
ond control units (1-1, 1-2, 1-m) for which the 
number of VCs exceeds a preset reference value 
to be over loaded. 

13. The device according to any preceding claim, 
wherein the load measurement means (6) meas- 
ures a response time between the commencement 
and termination of transmission for each of the sec- 
ond control units (1-1, 1-2 .... 1-m) on the basis of 
data relayed to it and notifies the distribution ratio 
adjustment means (7) of the result at regular inter- 
vals, and the distribution ratio adjustment means (7) 
further includes means for considering one of the 
second control units (1-1, 1-2 .... 1-m) to be over 
loaded for which the response time exceeds a pre- 
set reference value. 



14. The device according to any preceding claim, 
wherein the load measurement means (6) performs 
an operation determined by imitating the first control 
unit (5-1) in data sent by each of the second control 

5 units (1-1, 1-2 1-m), measures a service time re- 
quired to perform the operation, and notifies the dis- 
tribution ratio adjustment means (7) of the average 
service time, and the distribution ratio adjustment 
means (7) further includes means for deciding that 

10 one of the second control units (1 -1, 1 -2 1-m) is 
over loaded for which the average service time is 
larger than a preset reference value. 

15. The device according to claim 14, wherein the dis- 
15 tribution ratio adjustment means (7) further includes 

means for seeking the average of measured values 

of loads on the second control units (1-1, 1-2 

1 -m) and then determining the numbers of slots al- 
located for the second control units (1-1, 1-2 

20 1 -m) whose measured values differ from the aver- 
age by width 6 according to the ratio among their 
distances from the average. 

16. The device according to claim 15, further compris- 
es ing failure monitor means (8) for performing a ping 

command to each of the second control units (1-1 , 
1-2, .... 1-m) at regular intervals, decides that the 
second control units .(1-1, 1-2 1-m) which causes 
a certain number of time-outs is in the stopped 
30 state, and notifies the distribution ratio adjustment 
unit (7) of it. 

17. The device according to claim 16, further compris- 
ing a unit consisting of a group of the second control 

35 units (1-1, 1-2 1-m) of multisegment configura- 
tion to which another network is connected. 

18. The device according to claim 16, wherein the fail- 
ure monitor means (8) further includes means for 

40 trying whether it is possible to open a service port 
to transmit information from each of the second con- 
trol units (1-1 , 1-2, 1-m), deciding, after a certain 
number of failures, that the transmit information is 
being stopped, and notifies the distribution ratio ad- 

45 justment means (7). 

19. The device according to claim 18, wherein one of 
the second control units (1-1, 1-2, .... 1-m) includes 
the network load balancing device (3). 

so 

20. The device according to claim 18, further compris- 
ing another communication path directed from one 
of the second control units (1-1, 1-2 .... 1-m) to the 
first control unit (5-1 ) in addition to a communication 

55 path that passes through the network load balanc- 
ing device (3). 

21. The device according to any preceding claim, 
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wherein the first control unit (5-1 ) is a client compu- 
ter connected to a network, the second control units 
(1 -1 , 1 -2, . . 1 -m) are server computers connected 
to the network, and the conversion information stor- 
age means (4-a) is a conversion table from the first s 
control unit (5-1) to the second control units (1-1, 
1-2, .... 1-m). 

22. A network local balancing method for use with a 
system in which data is transferred from a first con- 10 
trol unit to a plurality of second control units over a 
network, said network load balancing method, com- 
prising the steps of: 

storing conversion information which repre- IS 
sents a correspondence relationship of the first 
control unit (5-1) to one of the second control 
units (1-1, 1-2, 1-m); 
transferring the data sent by the first control unit 
(5-1 ) to one or more of the second control units 20 
(1-1, 1-2, .... 1-m) according to the correspond- 
ence relationship in the conversion information 
stored by the storing conversion information 
step; 

adjusting the correspondence relationship in 2s 
the conversion information stored by the storing 
conversion information step; and 
measuring the load on each of the second con- 
trol units (1-1, 1-2, .... 1-m) and adjusting the 
correspondence relationship in the conversion 30 
information according to the load conditions 
each ot the second control units (1-1, 1-2, .... 
1-m). 

23. A network bad balancing program for use with a 35 
system in which data is transferred from a first con- 
trol unit (5-1 ) to a plurality of second control units 
(1-1,1-2 1-m) over a network, a computer-read- 
able recording medium which is recorded with the 
network load balancing program which allows a 40 
computer to perform; 

storing conversion information which repre- 
sents a correspondence relationship of the first 
control unit (5-1 ) to one of the second control *s 
units (1-1. 1-2, 1-m); 
transferring the data sent by the first control unit 
(5-1 ) to one or more of the second control units 
(1-1, 1-2, .... 1-m) according to the correspond- 
ence relationship in the conversion information so 
stored by the storing conversion information 
procedure; 

adjusting the correspondence relationship in 
the conversion information stored by the storing 
conversion information procedure; and ss 
measuring the load on each of the second con- 
trol units (1-1, 1-2, .... 1-m) and adjusting the 
correspondence relationship in the conversion 



information according to the load conditions 
each of the second control units (1-1, 1-2, 
1-m). 

24. In a network load balancing system which compris- 
es a first control unit which sends data, a plurality 
of second control units (1-1, 1-2, 1-m) which re- 
ceives the data, and a third control unit which trans- 
fers the data sent by the first control unit (5-1 ) to 
one of the second control units (1-1, 1-2, .... 1-m), 
the third control unit comprising: 

conversion information storing means (4-a) for 
storing conversion information which repre- 
sents a correspondence relationship of the first 
control unit (5-1) to each of the second control 

units (1-1, 1-2 1-m); 

distributive relay means (4) for transferring the 
data sent by the first control unit (5-1 ) to one or 
more of the second control units (1-1, 1-2, .... 
1 -m) according to the correspondence relation- 
ship in the conversion information stored in the 
conversion information storing means (4-a); 
distribution ratio adjusting means (7) for adjust- 
ing the correspondence relationship in the con- 
version information stored in the conversion in- 
formation storing means (4-a); and 
load measuring means (6) for measuring the 
load on each of the second control units (1-1 , 
1-2, .... 1-m) and notifying the distribution ratio 
adjusting means (7) of load conditions of the 
second control units (1-1 . 1-2, .... 1 -m) in order 
to allow the distribution ratio adjusting means 
(7) to adjust the correspondence relationship in 
the conversion information stored in the con- 
version information storing means (4-a). 

25. A balancing device for use with a system in which 
data is sent from a first control unit (5-1 ) to a plurality 
of second control units (1-1, 1-2, 1-m), said bal- 
ancing device comprising: 

conversion information storing means (4-a) for 
storing conversion information which repre- 
sents a correspondence relationship of the first 
control unit (5-1) to each of the second control 

units (1-1, 1-2 1-m); 

distributive relay means (4) for transferring the 
data sent by the first control unit (5-1 ) to one or 
more of the second control units (1-1, 1-2, 
1 -m) according to the correspondence relation- 
ship in the conversion information stored in the 
conversion information storing means (4-a); 
distribution ratio adjusting means (7) for adjust- 
ing the correspondence relationship in the con- 
version information stored in the conversion in- 
formation storing means (4-a); and 
load measuring means (6) for measuring the 
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load on each of the second control units (1-1, 
1-2, .... 1-m) and notifying the distribution ratio 
adjusting means (7) of load conditions of the 
second control units (1-1, 1-2, .... 1-m) in order 
to allow the distribution ratio adjusting means 5 
(7) to adjust the correspondence relationship in 
the conversion information stored in the con- 
version information storing means (4-a). 

26. A control device for use with a system in which data 10 
is sent from a first control unit (5-1) to a plurality of 
second control units (1-1, 1-2, .... 1-m), said control 
device comprising: 

conversion information storing means (4-a) for 15 
storing conversion information which repre- 
sents a correspondence relationship of the first 
control unit (5-1) to each of the second control 

units (1-1, 1-2 1-m); and 

distributive relay means (4) for transferring the 20 
data sent by the first control unit (5-1 ) to one or 
more of the second control units (1-1, 1-2, 
1 -m) according to the correspondence relation- 
ship in the conversion information stored in the 
conversion information storing means (4-a). 25 

27. A network load balancing for use with a system in 
which data is sent from a client to a plurality of serv- 
ers, said network load balancing device comprising: 



means for storing relevant data so that it will be 
processed in a corresponding one of the serv- 
ers; and 

means for deciding whether each of the servers 
is over loaded and, if it is, balancing the load on 
the servers so that it will be distributed to the 
other servers. 



30 



40 



45 



SO 



55 



18 



BNSOOCID: <EP 090390 1A2_t_> 



EP 0 903 901 A2 




19 



8NSOOCID: <EP 0903901A2J_> 



EP 0 903 901 A2 




20 



BNSDOCID: <EP 0903901 A2J_ 



EP 0 903 901 A2 




EP 0 903 901 A2 




EP 0 903 901 A2 



C START ) 



RECEIVE PACKET 



OBTAIN PATTERN / 



S 301 
S302 



SEARCH THROUGH PATTERN HASH TABLE FOR / 
MATCHING PATTERN 



S303 



APPLY PATTERN HASH FUNCTION 



OBTAIN SLOT HAVING HASH VALUE 



v S304 
SI 




S307 



CREATE VCP 



SEND PACKET TO SERVER 
IDENTIFIED BY VCP LIST 



SEND PACKET TO SERVER 
IDENTIFIED BY SLOT 




S308 



DELETE VCP 



( END j 

F I G. 5 



BNSOOCID: <EP 0903901 A2_l_> 



23 



EP 0 903 901 A2 





(*, *. 


2 3) 




0 


SERVER 


5- 


1 


1 


SERVER 


5- 


2 


2 


SERVER 


5 - 


3 


3 


SERVER 


5- 


1 


4 


SERVER 


5 - 


2 


5 


SERVER 


5 - 


3 


6 


SERVER 


5- 


1 


7 


SERVER 


5- 


1 



F I G. 6 



BNSDOCID: <EP 0903901 A2_l_> 



24 



EP 0 903 901 A2 





(*, *, 


2 3) 




0 


SERVER 


5 - 


1 


1 


SERVER 


5 - 


2 


2 


SERVER 


5 - 


3 


3 


SERVER 


5 - 


1 


4 


SERVER 


5 - 


2 


5 


SERVER 


5 - 


3 


6 


SERVER 


5 - 


1 


7 


SERVER 


5 - 


1 




F I G. 7 



BNSDOCID: <EP 0903901 A2_l_> 



25 



EP 0 903 901 A2 





(*. *, 


2 3) 




0 


SERVER 


5- 


1 


1 


SERVER 


5- 


2 


2 


SERVER 


5- 


3 


3 


SERVER 


5- 


2 


4 


SERVER 


5- 


2 


5 


SERVER 


5- 


3 


6 


SERVER 


5- 


1 


7 


SERVER 


5- 


1 




FIG. 8 



RNSnnttD: <EP 0903901 A2 I > 



26 



EP 0 903 901 A2 













CVJ 






t 

lo 






! 






! 

O 






CL 






o 






CM 

i 


















o 














s 






1 






to 


















o 






o 


















CL 












o 













O 
LL 







CM 


CO 


CM 


CM 


co 






>— N. 


1 


1 


1 


1 


1 


1 


1 


1 


CO 
CM 


m 


lo 


lo 


to 




LO 


LO 


LO 




CC 


Of 


or 


a: 


GL 


a: 


ce 


* 


UJ 


UJ 


UJ 


UJ 


UJ 


UJ 


UJ 


UJ 


> 


> 


> 


> 


> 


> 


> 


> 






<r 


(X 


a: 






a: 


cr 


UJ 


UJ 


UJ 


UJ 


UJ 


UJ 


UJ 


UJ 




CO 


CO 


CO 


CO 


CO 


CO 


CO 


CO 




CD 




CM 


rj 




LO 


to 





t 



BNSDOCIO: <EP 0903901 A2J_> 



27 



EP 0 903 901 A2 





(*, *, 


2 3) 




0 


SERVER 


5- 


1 


1 


SERVER 


5- 


2 


2 


SERVER 


5- 


3 


3 


SERVER 


5- 


2 


4 


SERVER 


5- 


2 


5 


SERVER 


5 - 


3 


6 


SERVER 


5- 


1 


7 


SERVER 


5- 


1 




FIG. 10 



BNSDOCID: <EP 090390 1A2_L> 



28 



EP 0 903 901 A2 



c 



START 



) 







MEASURE LOAD Oil EACH SERVER 






NOTIFY DISTRIBUTION RATIO 
ADJUSTING UNIT OF RESULTS 







Sill 



S112 



( ENP ) 



FIG. 11 



29 



EP 0 903 901 A2 



(£> 



c 



START 



ail 



RECEIVE NOTIFICATION OF LOAD CONDITIONS 



1/ 




NOT OVER LOADED 
IS i-th NOR STOPPED 

SERVER OVER LOADED OR 
STOPPED ? 



V 



(end] 



DETERMINE DECREMENT ADJUST- 
MENT QUANTITY SO AS NOT TO BE- V 
COME MINIMUM NUMBER OF SLOTS 



MAKE NUMBER OF 
SLOTS 0 




ADJUST- 
MENT QUANTITY * 0 

? 

"res 

$91 7 M DECREASE NUMBER OF SLOTS 
BY ADJUSTMENT QUANTITY 



SET OVER LOAD FLAG ON 



No 



No 



ARE THERE 
SERVERS NOT OVER LOADED NOR, 
STOPPED ? 

fes 

"ARE THERE" 
SERVERS FOR WHICH SLOTS CAN BE 
INCREASED ?_ 
Yes 



© © 



© 



FIG. 12 



30 

BNSOOCID: <EP 0903901 A2_L> 



EP0 903 901 A2 



RESTORE NUMBER OF 
SLOTS TO ORIGINAL 
NUMBER 




SELECT ONE SERVER FROM 
SERVERS SELECTED IN 
STEP S920 



7 



SELECT ONE SERVER r 



S924 



DETERMINE MINIMUM NUMBER OF SLOTS 



DETERMINE INCREMENT ADJUSTMENT QUANTITY SO 
AS NOT TO BECOME LARGER THAN MAXIMUM NUMBER 



INCREASE NUMBER OF SLOTS 



© 




No 









i + 1 



SET EXPERIENCE FLAG OFF FOR ALL SERVERS 



S934 



DECREASE DECREMENT ADJUSTMENT QUANTITY / 



FIG. 13 



6 



31 



BNSOOCIO: <EP 090390 1A2_I_> 



EP0 903 901 A2 




32 

BNSDOCID: <EP 0903901A2J_> 



EP 0 903 901 A2 



if) 




BNSDOCID: <EP 0903901A2J_> 



33 



EP 0 903 901 A2 



*- MAXIMUM NUMBER OF SLOTS 



ADJUSTMENT 
QUANTITY 3 



_ MINIMUM NUMBER 
OF SLOTS 



NUMBER OF SLOTS 
FOR SERVER OVER 
LOADED 



NUMBER OF SLOTS 
SERVER NOT OVER 
LOADED 



FOR 



FIG. 16 



34 



EP 0 903 901 A2 




BNSDOCID: <EP 0903901 A2_l_> 



35 



EP 0 903 901 A2 




UJ = 




BNSDOCID: <EP 0903901 A2_l_> 



36 



EP 0 903 901 A2 



n 




SERVER No. 



OVER LOAD 

EXPERIENCE 

FLAG 



1 
(OVER 
LOATED) 

ON 



2 
(OVER 
LOADED) 

OFF 
I 

ON 



ON 



4 

OFF 



VARIATIONS 
IN NUMBER 
OF SLOTS 



DECREASED 
BY n 



DECREASED 
BY m 



INCREASED 
BY n 

INCREASED 
BY m 



FIG. 19 



BMSDOCID: <EP 0903901 A2_l_> 



37 



EP 0 903 901 A2 




msnr-min- <fp 0903901 A2 l > 



38 




BNSDOCID: <EP 0903901A2_I_> 



39 



EP 0 903 901 A2 




40 

BNSDOCID: <EP 0903901 A2_l_> 



EP 0 903 901 A2 



CO — 

oc h- 

LU CJ> 



CO O 









LU 
> 


0£ 




oe 


LU 




UJ 


=>• 




CO 


Cm! 
LU 




oc 


CO 




o 






LL_ 


OH 






<z> 




LU 


LL_ 










_J 


LU 




-< 






>• " 


— 1 










(XJ OC 


> 




O "J 






z >- 


LU 




UJttO 




ac LU 


= 




LU CO 


LU 




u. 






LU OC 


LU 


OS 


oco 


U- 


LU 


u- 

LU 

3 CO 


LU 
Ck£ 


ERV 




CO 


CO 




z 

o 


OC 




CTI 


FO 


gs 


LU 


AG 


i° 


COI 


— i 

u. 




u_ 


o 


o 


o 


-< 


UJ 


1 


o 




oc 


— 1 


lu 


LU 


1 


Of GO 


CO 


OS 






LU 


:> = 




> 


-<3E 







- - — — All VII 



oo ooo 



< 

00 



7 











/ °\ 












>» \ 






v \ 




l w / 


c \ 


w 


CO I Q) / 


O \ 


1 ° 


z * / 




\ ^ 




c 




VII \ 








\— \ 






C-5 / 






« / 






I X / 






\ s*\ I 












\ °/ 





CO 
CNJ 

o 

Ll 



41 

BNSDOCID: <EP 0903901A2_I_> 



EP 0 903 901 A2 




EP 0 903 901 A2 



ti: MEASURED VALUE SENEh 

■ & 

OBTAIN AVERAGE A OF tx 

. 



FOR ALL tx 




yes 

± 

n«- Sx/tx* (A-tx) 



± 

Sx^-Sx + n 



FIG. 2 5 



43 



EP 0 903 901 A2 



DECREMENT 
ADJUSTMENT 
QUANTITY 3 



NUMBER OF 
SLOTS FOR 
SERVER 
OVER 
LOADED 



INCREMENT 
^ADJUSTMENT 
\QUANTITY 1 



INCREMENT 
ADJUSTMENT 
QUANTITY 2 



0 



NUMBER OF SLOTS FOR 
TWO SERVERS NOT 
OVER LOADED 



FIG. 26 



44 



EP0 903 901 A2 



SLTi* (l-o)*SLTi 
SLTj - SLTj + n*SLTi 






SLTi — SLTi' 




s. 


FIG. 


27 



RNSDOCID. <EP 0903901 A2_l_> 



45 



EP 0 903 901 A2 



O 
Id 
I 
O 



UJ 
Q_ 
O 




CD 
C\J 



o 
o 

LU 
O 

< 

j.. 



46 

BNSOOCID: <EP 0903901A2_I_> 



EP 0 903 901 A2 



FIG. 29A 



CjvjETWORK^) 



5-1 5-2 5-n 
SERVER 



FIG. 29B 



CJetwork^) 



3 



5-1 5-2 5-n 
SERVER 



FIG. 29C 



5-1 5-n 
SERVER 



47 



EP 0 903 901 A2 




BNSOOCID: <EP 090390 1A2_L> 



48 



(19) 



J 



Europaisches Patentamt 
European Patent Office 
Office europeen des brevets 



(12) 



(11) EP 0 903 901 A3 

EUROPEAN PATENT APPLICATION 



(88) Date of publication A3: 

20.07.2005 Bulletin 2005/29 

(43) Date of publication A2: 

24.03.1999 Bulletin 1999/12 

(21) Application number: 98302410.0 

(22) Date of filing: 30.03.1998 



(51) Intel 7: H04L 12/56, H04L 29/06 



(84) Designated Contracting States: 


• Hosoi, Akira 


AT BE CH DE DK ES Fl FR GB GR IE IT LI LU MC 


Nakahara-ku, Kawasaki-shi, Kanagawa 211 (JP) 


NLPTSE 


• Yamada, Takuya 


Designated Extension States: 


Nakahara-ku, Kawasaki-shi, Kanagawa 211 (JP) 


AL LT LV MK RO SI 


• Koide, Masuhiro, c/o Fujitsu Shizuoka Eng., Ltd. 




Shizuoka-shi, Shizuoka 422-8067 (JP) 


(30) Priority: 22.09.1997 JP 25739397 






(74) Representative: Stebbing, Timothy Charles et at 


(71) Applicant: FUJITSU LIMITED 


Haseltine Lake & Co., 


Kawasaki-shi, Kanagawa 211-8588 (JP) 


Imperial House, 




15-19 Kingsway 


(72) Inventors: 


London WC2B 6UD (GB) 


• Takahashi, Eiichi 




Nakahara-ku, Kawasaki-shi, Kanagawa 211 (JP) 





CO 

< 

© 

O) 
CO 

o 
o> 

o 

Q. 
LU 



(54) Network service server load balancing 

(57) For use with a system in which data is trans- 
ferred from a first control unit (5-1) to a plurality of sec- 
ond control units (1-1 , 1-2 1-m) over a network (2), 

a network load balancing device (3) comprises: a con- 
version information storage unit (4-a) which stores con- 
version information representing a correspondence re- 
lationship of the first control unit (5-1 ) to one of the sec- 
ond control units (1-1 , 1-2, 1-m); a distributive relay 
unit (4) which transfers data sent by the first control unit 
(5-1) to one or more of the second control units (1-1, 
1-2, .... 1-m) on the basis of the correspondence rela- 
tionship stored in the conversion information storage 
unit (4-a); a distribution ratio adjusting unit (7) which ad- 
justs the correspondence relationship in the conversion 
information stored in the conversion information storage 
unit (4-a); and a load measuring unit (6) which measures 
the load on each of the second control units (1-1, 1-2, .... 
1-m) and notifies the distribution ratio adjusting unit (7) 
of the measurements to allow the unit to adjust the cor- 
respondence relationship in the conversion information 
stored in the conversion information storage unit (4-a). 



tUHT COMTEK 



1-2 

X 



E T W O F* K 



3. 




DWI (BUT 101 IATI0 
ADJUST IRC UNIT 



r 


1 


LOAD 




FA HIKE 


REASUPE- 




U0NITG8 


MEJJT WIT 




WIT 



CORTtOL 
WIT 



STATUS 
WIT 



NETWORK SERVICE SERVES LOAD BALANCING DEVICE 



USER 



ii 



F I G. 2 



SERVES CHHTEK 



Printed by Jouve, 75001 PARIS <FR) 



BNSDOCID: <EP 0903901 A3_l_> 



EP 0 903 901 A3 



European Patent 
Office 



EUROPEAN SEARCH REPORT 



EP 98 30 2410 



DOCUMENTS CONSIDERED TO BE RELEVANT 



Category 



Citation of document with indication, where appropriate, 
of relevant passages 



Relevant 
to claim 



CLASSIFICATION OF THE 
APPLICATION (lnlCI.6) 



EP 0 648 
MACHINES 
BUSINESS 

* column 

* column 

* column 

* column 

* column 

* column 

* column 

* column 

* column 

* claims 

* figures 



038 A (INTERNATIONAL BUSINESS 
CORPORATION; INTERNATIONAL 
MA) 12 April 1995 (1995-04-12) 
1, line 1 - line 12 * 
1, line 32 - line 36 * 
line 6 * 

- line 50 * 

- column 4, line 51 * 

- line 39 * 

- column 7, line 7 * 
column 10, line 37 * 



1-27 



H04L12/56 
H04L29/06 



2, 
2, 
2, 
5, 
6, 
9, 
11, 



line 
line 
line 
line 
line 20 
line 40 
line 



2 - 
46 
56 



9 - 

line 47 - column 12, line 26 



1,3,5,7,9,12 * 
1-3 * 



US 5 548 724 A (AKIZAWA ET AL) 
20 August 1996 (1996-08-20) 

* column 1, line 15 - line 20 * 

* column 2, line 1 - column 3, line 41 * 

* claims 1-3,5,8,9,14-16,18,20 * 



1-27 



TECHNICAL FIELDS 
SEARCHED (lnLCI.6) 



H04L 



The present search report has been drawn up for all daim3 



Place erf aeaich 

Munich 



Date of conipfeUon of the seaich 

10 May 2005 



Examine* 

Schrembs, G 



CATEGORY OF CITED DOCUMENTS 

X : particularly relevant if taken atone 

Y : particularly relevant if combined with another 

document of the same category 
A : techno log (oai background 
O : non-written disclosure 
P: 



T : theory or prinople underlying the invention 
E : earlier patent document, but published on, or 

after the filing date 
D : Document cited in the application 
L : document cited for other reasons 

& : member of the same patent family, corresponding 
document 



BNSDOCID <EP 090390 1A3_I_> 



EP 0 903 901 A3 



ANNEX TO THE EUROPEAN SEARCH REPORT 

ON EUROPEAN PATENT APPLICATION NO. EP 98 30 2410 



This annex lists the patent family members relating to the patent documents cited in the above-mentioned European search report. 
The members are as contained in the European Patent Office EDP file on 

The European Patent Offioe is in no way liable for these particulars which are merety given for the purpose of information. 

10-05-2005 



Patent document 


Publication 




Patent family 


Publication 


cited in search report 


date 




members) 


date 


EP 0648038 A 


12-04-1995 


GB 


2281793 A 


15-03-1995 






DE 


69433833 Dl 


15-07-2004 






EP 


0648038 A2 


12-04-1995 






JP 


2505116 B2 


05-06-1996 






JP 


7093238 A 


07-04-1995 






US 


6282569 Bl 


28-08-2001 



US 5548724 A 20-08-1996 JP 6332782 A 02-12-1994 



For more details about this annex : see OfficialJoumal of the European Patent Office, No. 12/82 



3 



0903901 A3 J_> 



